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David C. Berliner 
me ; 
The University of Arizona 
The stuff of the Be,inning Teacher Evaluation Study proposed to in- 
‘yvestigate elementary school reading and mathematics instruction in a slightly 
novel way. They made a simple modification of the process-product approach . 4 
to the study of classroom learning. The modification is based on the be- . 


lief that what a teacher does at any one moment in time while working in a 


circumscribed content area affects a Student primarily at only that particu- 

lar moment in time and di thant particular content area. iieeekare, the oe 
link between teacher behavior and student achievement is the ongoing student 
behavior in the classroom learning situation. The logic Sontinues in this 

way. What a teacher does to foster learning in a particular content area, 

becomes important only if a student is engaged with appropriate curriculum 

content. Appropriate curriculum content is defined as curriculum that is- 


logically related to the criterion and is of an easy level of difficulty 


for the studcrit. Thus, a second-grade student engaged in the task of two 
column addition, without regrouping, either by means of a workbook or by 
wacchinu: the teacher at the chalkboard, is engaging in processes that can 
twe column addition 
lead to proficiency in deceding-—btends, if the task requires a:low error 
rate on the part of the student. The variable used for research purposes 
is the accrued engaged time in a particular content area with en ae 
that are of an casy level of difficulty. This complex variable is called 
Academic Leurning Time (ALT). hough pkobably not linear, the accrual of 


= ALT is expected tu be u strong positive correlate of achievement. 


Error Rate_and ALT 

Our aeke tind belief was that engagement with curriculum materials of 
an intermediate kevel of difficulty would lead to greater achievement. Our 
data, however, have convinced us that pintine children trying to learn mathe-. 
matics in traditional clussroom settings need to work on academic tasks that 
give rise to low error rates for the students. A low error rate occurs when 
about 20 percent or fewer errors are noted for a student engaged in workbook 
pages, tests or classroom exercises. When a student's responses are not 
overt, an observer must estimate the level of difficulty of the activities 
in which the student is ensupeds 

Certainly trying to keep a student engaged for too long with too many 
easy mathematics tasks will not help a student's academic performance. En- 
gagement is likely to drop off, and content coverage will be minimal. 

A teacher must know when to move a student to new materials and activities. 

_ This is a very complex diagnostic decision that teachers must frequently 
make. “But with proper student preparation, clear explanations, appropri- 
ate structure and scquencing, even the new material to be learned can pro- 
duce a low error rate. In the conception of. classroom learning proposed 
here, it is when teachers put students into contact with mathewatios curricu- 
lum materials und activities of an casy level. of difficulty that learning ie 
hypothesized to take place. 

The variable of ALT, which is measured in real time, has some roots in. 
the work of Carroll (1963), Bloom (1976), Harnischfceger and Wiley (1976) and 
others. ‘The cffort to develop this variable, which focuses on student 
use of time and student curriculum, Simultaneously, also stems from the 


extant literature concerned with research on teaching. In that literature 


. 3 
(Berliner {| Rosenshine, 1977; Rosenshine & Berliner, 1978) a vector of vari- 
ables which ive cdlled direct instructional variables ‘seemed to consistently 
show relations with student academic achievement. The students’ use of time 
and curriculum materials are part of that vector. ‘And finally, the concern 
_ £or the easy level of difficulty of the curriculum materials comes from our 
own pilot data and the instructional design literature. That literature 
emphasizes the importance of practice, repetition and overlearning for reten- 
tion, und the utility of small steps and low error rates in learning with . 
programned instructional materials. With ALT a central focus of siakeecon 
research the: typical process-product paradigm for research on teaching must 


be modified. ‘This modification may be schematized as in Figure 1. 


, Insert Figure 1 about here 


a en Ee Re 


In this conception of research on teaching the content area the stu- 
dent ia working on must be specified precisely, student task engagement must 
be judged, the level of difficulty of the task must be rated, and time must 
‘be measured. The constructed variable of ALT, then, stands between measures. 
of- teaching. and measures of seadent achievement. A design for research | 
using this approach requires the construction of two correlational matrices. 


The first is used: to study how teacher behavior and classroom characteris- 


tics affect ALT. The second is uy to study how ALT and achievement are . 
related. | te | | > ig 

In this conception of how teachers influence student achievement tho 
variable of engaged time is emphasized. Engaged time is the upper limit 
for ALT. And, the upper limit on asasures of engaged time in classrooms, - 


for a particular content area, is the time the teacher has allocated for 


“yj 
instruction in that content area. -Il.ec remainder of this paper is concerned 
with ai tecatod time, engaged time, and academic learning tipe (ALT) in differ- 
ent content areas. ‘The ‘thesis of this paper is that the marked variability 
in allocated ‘time, engaged time, and in ALT, between and within classes, is 
the most potent cxplanutory variable to account for variability in student 
achievement, after initial aptitude has been penaved as a predictor variable. 
A corollary of this thesis is that interactive teaching behavior (praise, © 
questioning, use of organizers, feedback, etc.) can only be understood © 
through their effects m ALT. In this conception of classroom learning the 
_interactive teaching behaviors or teaching skills are not thought to be 


directly linked to achievement. 


A Study of Instructional Time 

Allocated time, engaged time, and ALT were studied during a recent 
school year in approximately 25 second and 25 fifth grade California classes. 
Teachers were trained in log-keeping procedures so that the daily timo allo- 
cations for selected students seuld*be recorded within particular content 
areas of reading and mathematics. © In addition, a trained observer was pre- 
sent approximately one day a week foci 20 weeks of the school year. 
The observer recorded engaged time and provided data to compute estimates 
* of ALT as well as providing data about a number of other facets of class- 
room life (Beginning Teacher Evaluation Study, 1976). Selections from this 
larger data set (Dishaw, 1977a; Dishaw, 1977b; Filby & Marliave, 1977) will 
be used to illustrate some of the within and between class variability in . 
allocated time, engaged time, and ALT. Only the data on mathematics instruc-. 


tion will be presented in this paper. . 


Allocated Time — : 


Table 1 presents allocated time in content areas of second-grade mathe- 
matics and Table 2 apobonts allocated time for content areas of fifth-grade 
mathematics. These data were obtained from teachers' logs over an average 
of 90 days instruction from October to May of the school year. The logs. 
were Filled out daily for six students in each of the classes. Within each 
grade level, the students were of compurable ability levels ‘6 reading and 
mathematics, both within and across classes (Cahen, 1977). The data from 


the six students that were studied intensively in each class will eae 


“to generalize about the whole class. 


Insert Tables 1 and 2 -here 


With the data from both second and fifth grade mathematics one can 


*. ° 


notice widespread variability in how teachers spend their time. Different 


philosophies of education result in different beliefs about what is impor- © 
tant for students to learn. These béliefs, along with the teacher's likes 
and dislikes for teaching certain areas, result in some interesting differ- 
ences in the functional curriculum of a ass: For example, from Table 1 
it can be seen that ‘students in class 13 had an average of 400 minutes 

each to learn the concepts and operations involved in linear measurement, 
while students in class S had an average of 29 minutes each to learn chess’ 
operations and concepts. In the content area of fractions and in the con- 
tent urea of money class 21 received very little time while class 13 


received markedly more time in these areas. From Table 2 it can be seen 


8 


that classrooms 14 and 18 spend dramatically more time on division than 


the other two fifth-grade classes. In classroom 18 fractions were 


¢ 
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, ? 


ri i 4 . 
emphasized, as ‘judged from the dramatically greater allocation of time to’ 
r : 


° 


that content area, in contrast to the average amount of time each student of 
class 3, 4, and 14 received. And word problems hardly seemed to be of 


interest to the teachers of dlasses 3, 4, and 18, at least that is what can 


be concluded when the data from these three classes are compured with the - 


data from class 14, 

These rather significant differences in the functional classroom 
curriculum should, by all we know about learning, result in considerable 
differences in achievement. If students in these second grade classrooms 
‘were tested at the end of the year on linear measurement, you might do well 
to wager that students in cies 13 would demonstrate better performance 
than students in class 5. If these fifth grade classes were part of some 
end-of-year statewide testing program, where fractions were tested, as it 
often is, one might well expect that students in classroom 18 would show 
superior performance when contrasted to similar students in the other fifth 
grade classes. 

The broad-spectrum standardized achievement test in mathematics may be 
a sovial indicator, from which state or national policy can be illuminated. 
But as long as teachers have the freedom to choose what areas th will 
emphasize in their classrooms, these tests can never be used as fair mea- 
sures of teacher effectiveness. It simply is not fair to teachers to 
“evaluate their students in areas that the teacher did not cover or empha- 
size. On the other hand, it may not be fair to students and their parents 
to let teachers arbitrarily choose that which is to be taught. rr tighter 
control of the functional. classroom curriculum may be désirabie, This prob- 


lem is recognized by many, and has led some cugriculum developers to insist 


¢ 


8 ; 


certain minimum number of minutes pér day or hours per week for certain, 


t . : ‘ 


upon aceaigene control of teacher behavior in Onder to implemenv the program po 
that they ‘a promote (c.g., Becher § Engelmann, 1978). 


Another interesting aspect, of allocated time is the average daily 
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’ 


time Gevoted to mathematics. States and school districts often mandate a 


subject matters: Let us suppose, by law, that 40 minutes a day is the mini- 


mum tiny to be devoted to mathematics in the second grade, BAC: a particu- 


lar ie district. Let us also suppose that this mathematics time begins 
at 11:15, after a recess, and that the time neriod devoted to mathematics 
ends at noon. The teacher, principal, and superintendent may well feel that 
the state minimum requirements are being met and exceeded. But careful 
observation will reveal otherwise. A 10-minute delay in the start of the 
work, called transition time in the Beginnin ‘Teacher Evaluation Study, may 
occur’ before the mathematics curriculum is feally in offect. Toward the 3 
end of the allocated time students are putting workbooks, contracts, and 

cuisinaire rods away, getting lunches out, and lining up for the noontime 
dismissal. Another ten minutes may be/lost. Functional time for mathe- 

matics is now 25 minutes, which is 09 percent under the legal requirement. 


wn? : / : : r 
The data presented in Table Y reflect this difficulty in classroom \ 


management of time. Classes 5 and 21 have, on the average, a daily alloca- 


/ 


‘tion of mathematics totalling about 30 minutes per day, while clesses 8 and 


13 show, on the average, an allocation of time to mathematics of over 50 


minutes per day. From other data collected as part of this study’ we estimate 
i 

that the atone in class 5 and class 21) spend an. average of 42.5 minutes> 

per day ia brankitians rom | activity to activity, while oe in classes 


8 and 13 spend about half that time, approximately 22 minute per day, in 


/ 
/ 


| - / 
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trunsitions. The average daily minutes per day devoted to fifth grade mathe- 


*y 


“matics, as’ presented in Table show similar variability. Teachers in 


classes 14 and 18 have allocated over 100 percent more total vine to mathe- 


matics than did the teachers in classes 3 and 4. Other data from our full 


fifth grade sample reveals i shat the teachers with the lower rate of allocated 


tine had higher ‘hati. average class time “spent in 1 transitions ‘ana behavioral 


management, What this indicates is that the time allocdted for academic ine. - ° 


: 


struction in a school nay joan easily slip away when a teacher cannot keep 
the transitional time atid behavioral problems to a ninimun. Any sensible 
Manager knows that. i aa however, in many classes, there is a amine. 
about classroom management that results in considerable inefficiency. 

This brief examination of selected data presenting estimates of class- 
room allocated time shows clearly that some teachers spend considerable more 
time instructing in particular content areas than other teachers, and some 
teachers allocate considerably more total’ instructional. time than do other 
. teachers. These differences, put into experimental terminology, Henresent 
clear differences in the type and in the duration of treatment. One can 
expect, therefore, considerable variability on the outcome measures used to 
assess: these vastly different treatments. | 

One other insesustioual variable of interest. to an instructional 
designer is sequence of instruction. Not only. should type and duration of . 
‘instruction affect learning outcomes, but the sequencing of the instruction 
should also affect what is learned. Do students in differont classrooms 


accumulate their time in the same content areas in different ways? Table 
3 presents data to answer this question. In this table the, raw allocated 


and cumulative allocated time for five second grade students receiving 


i0 


instruction in addition and subtraction (no regrouping) is recorded. “Student 
0506 received about 100 percent-more time in this oe ares than did the 
other students. ‘And he received it continuously through the 17 wecks of 
instruction. Student 0702 received his instruction in 10 weeks flat. No 24 


\ 
review or further instruction were noted in this content area. Student 1006 


received two-thirds of her instruction in 9 woaks, but had one-third more ; 

instructional time in this content area allocated from weeks 10-17. These 
* vastly different sequences of instructional time allocations are, from an 

instructional design standpoint, quite important. They are systematically 


uninvestigated variables in most research on teaching 


e 


aaa tt ae ae 
Insert Table 3 about here. 


Engaged Time ” aa . ~ 
. Tables 1 and 2 also present data on the average percent of time stu- 
iii are engaged in mathematics instruction. These data are from observer 7 
records, and not from teacher logs. Previous work revealed that teachers 
can keep accurate records of allocated time, but that classroom observers 
were necessary to obtain accurate records of snagged tine “(Werliave, Filby, 
& Fisher, 1976). In examining these data it appears that the percent of 
time students arc enawued is relatively high. This is an artifact of the 
cbservattonai system that was in use. . The observation system required 
“that transition time and ether blassnods phenomena be coded as separate 
events. Thus, the data on engagement rates are for the time spent/in mathe- 


* 


matics, after a class has settled down and before the class starts to put- 


their work away. If engagement were coded for the entire time-block denoted 
by. teachers as mathematics time, the engaged time rates would be considerably 
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lower because duriny transitions most of the class-is not engaged. Still, 
! . ‘ a 


wefiabitity between classes is noted for a potentially” important variable.” = 
. d e - ; rug 
” che engagement rates in these four second-grade classes vary from 61 percent 


_to 78 percent during mathematics peek a In the four fifth-grade 


? ia VL ‘ 8 


=? 7 : x 
classes, engygement rates vary from 60 percent “to 80 percent during di 


' matics Ute “hese ranges were much larger in the total sample of 
: . oF 6 . i 


F classes that were: studied. 
! . The average number of minutes per day allocated for<instruction, multi-  —~ \ 


plied by te engagement rate, provides liberal estimates of the number of - 


‘ 


a 
3 engaged minutes per day, per, student. These data are found in Table ] and, 


Table 2. In the second-grade mathematics data, at the lower end of the. * 


. 
of 


‘Re- ° 
f 


range, 20 minutes of engaged time per day is noted~ At the higher end. pi 4 “ 
the range 40 minutes per day,is noted. For fifth-grade mathematics the" * 
z range in these, four. ee between 17 and 49 minutes per day of-engaged = =. : 
‘ time. These are dramatic differences, differences of 100% or. a in the 
‘ engaged time students allot to. learn their. mathematics. For reasons that : Z 
he ,2 we do not yet fully understand, some combination of teacher behavior and =—s 
students! s6cialization to school interact to produce classes where most | 
of the children are attending to their’ yore want of the time. And these | ‘ 
‘sume factors sometines result in classes where less than half ‘Of the chil- 


; 4 
; dren are attending ‘to their Siok during the time allocated for instruction, a 
In most districts we may assume that a school ear is about 180 days. 
This figure must be reduced by absences of teachers and @@udents, Strikes, , 
bussing difficulties, the difficulties of instruction before Christmas and ee 

| Easter breaks, the testing at the beginning and end of the school year, and” 


other factors. A reasonable estimate of the "functional" school year*may——*——- 


s f 
. re : ‘ 
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ll 
be about +50 ‘tases. eee the engaged minutes per day over these 150 
_ days gives” an estimate of the deeatied instructional time allotted by students 
to the academic curriculum dursne the entire school year. Tables 1 and‘2 
_ present these data for the four classes in each grade level. Between SO and 
100 hours per year of active stuitent involvement in classroom imathematics 
instruction is oted in the four se classes. In the four fifth- 
grade classes, even with more mature isk supposedly more independent learners, 
the range is between 43 and 123 sisiicte aie ‘per school year. 
As these data come to light some important questions must be asked. 
For example, what should be expected in the way of engaged time for 30 
Students and one teacher, weeks together throughout the school year? 
. What are the a el aa for. instructional time held by parents and school 
board members as they make policy. to educate the young of a communi ty? “Bee 
~" cause these new estimates of classroom allocated and engaged time do not 
goetone to the prevailing beliefs that exist among the people who diaize 
iss and support education, either those beliefs must/Re changed, or instruction 
al practices must be altered: 
Academic Learning Time | 
. -* As noted above, academic learning time is the research variable of 
most interest a _the Beginning zeatuet ontuakinn Study. Grid: SompaKeRt of 


‘ ALT is the level "af difficulty of the material that is attended to by a © 


student. It is the belief of the investigators that learning occurs pri- 


_ marily with materials of an easy difficulty level. ‘Materials that arc too 


a hard for a’ student do not add, much to his acquisition of the concepts, . 


ions s_ that are ‘required of ‘Students i in a particular grade 7 


_ Skills, and operat 


". Jevol, Materials that are.casy to master promote retention. High levels 


¢ 


Py ° 


. 
. ey e 
“ 
i . 
: ‘ 
° "J * 
’ 
4% . % 
. 


12 
of retention are needed for demonstration of achievement eatne in end of year 
testing nedprane Tables 1 and 2 present information on the percent of time 
that students are working with material of an easy level of difficulty. These 
data are ratings made by observers in classrooms. As shown in Table 1, for : 
second-grade mathematics, the range is beeen SS percent and 67 percent. In 
fifth-grade mathematics the range is between 41 percent-and 80 percent. | 
Multiplying the engaged minutes per day by the percent of time students are | 
assigned work of an casy level of difficulty provides an estimate of ALT per 
day. These duta are also provided in Tables 1 and 2 
As noted above, the typical academic school. year of 180 days may’ be 
sansideved to be a functional school year of 1S0 days. The last’ line in 
_ Table 1 and Table 2. presents academic learning time, in hours, for a school 
year of 150 days. tn these four-classes, at each grade level, digferénces 
of many hundreds of percent , in ‘accumilated ALT are noned. -Im second grade 
witnsanties: the range is from 30 hours per school year to 58 hours per 
school year. In fifth-grade wathenatica the range is from 18 risus oak 
“school year to 53 hours per school year. In the total sample studied the 
range of ALT is larger. It should also be noted that all the stoucntory: 


school teachers in this sample were volunteers. These data, if they could 


. be obtained: from a non-volunteer sample, would most likely show even more 


between class variability 


t? 


: If academic learning time is one key to acquiring the knowledge and 
skill required to master he curriculum of a particular grade level, for a 
. particular content arta one can sec that the school year anes not contain 


as much ALT as might be gested, if our concerns about Snetriiction are 


correct, there_are_many, many classes where there is not sufficient time _ 


‘ fod 


1 
’ 


j 
for students to master the curriculun which has been chosen for them. 
. ! e 
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Relations between the Three Time Variables 


One way to ‘think about the relations between allocated time, engaged 


we 


time, and academic learning time is to examine the bar graph presented in 
Figure 2. The three time variables were measured. in the content ‘area of 
place value in second grade mathematics. These data from five students pre- 
sent un interesting problem, whose implications will be understood better in 
future studies. If we rank order the students in térms of allocated time,’ 
the students would seling ‘cieewass if we rank ordered by engaged time, they 
would rank order a second way; ani if we rank ordered by ALT they would 
order a third way. In particular, Student three received the lowest ullo- 
cation of time ‘in place valuc, displayed the fourth pdotient level of engaged 
time, and showed the third highest level of ALT.: We think these changes in 


Magnitude of the three variables have important implications for classroom 


learning. But, we do not yet fully understand them. - ; t- 


Iasert Figure 2 about here 
; ‘Summary . | o 
Descriptive data on allocated time, engaged time, and academic learn- 
ing time’ have been presented The data from four second-grade and four 
fifth-grade classes, chosen to reflect differences in the variables of 
interest; were examined, If the type of treatment and the duration of 
treatment and the sequence of treatment are crucial variables in the deter- 
mination of what is teamed and how much is learned, then the between class 
differences in the weekly and total allocated time in content areas, and in 


_ total allocated time per day or per school year, become important operationally 
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defined behaoral indicators of the instructional treatment. If lcarning is 
likely te er only when students attend to the instruction offered them, 
then between class differences in engaged time become an important operation- 
ally detined behavioral indicator of the effective stimulus situation, as 
opposed to the nominal stimulus ‘situation. And finally, if learning pri- 
marily takes place when. students are engaged with materials and activities 
that are of an_casy level of difficulty for that particular student, then 
AUT becomes an important operationally defined behavioral indicator of 
student learning. The construct of ALT has an intriguing virtue, One does 
not need to wait until the end of the school year to decide if learning ‘has 
_taken place. One can study learning as it happens, if the construct of ALT 
GF accepted. as it has been defined. In the conception of instruction that 
has guided the research that has been wonaucted 4 ALT and learning are synony- 
mous. ‘ 
‘The commonckéiae logic of the above statements is appealing. Eupiri- 
cal SUASENES at this writing, is very MO CRU TPE ANE The ALT variables, in 
regression ‘tnaipsed;, are accounting for about 10 percent of the variance 
in mathematics achievement in the various content areas, after the effects 
of pretests -have been removed. This is quite a lot. Both logic and — 
empirical data urge us, to examine seriously the eaiecoe allocated time, 
engaged time, and ALT in promoting achievement. Such concerns can lead 
teachers and supervisors of teachers to examine eiassroon processes in ways 
that logically relate to student achievement. Without turning classes into 
" authoritarian factories of learning, many teachers can improve their effec- 
tiveness by attending to these variables and reorganizing classroom piactiee 
to maximize teaching time and learning time--resources over which they have 


ere tere arena 9 -ceetenveetitenntatenratarnnreangenne 


considerable personal control. 
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The ideas and data presented in this paper emerged from work performed 
while conducting the Beginning Teacher Byaluation Study. ‘That study was 
funded by the National Tivtituke of Education and udministered by the 
California Commission for Teacher. Preparation and Licensing. The research 
was conducted by the Far West Laboratory for Educational Research and 
Development. The study has been a joint effort by David C. Berliner, 
Leonard S. Cahen, Nikola N. Filby, Charles W. Fisher, Richard N. Marliave, 


and Jeffry E. Moore. 
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Table 3 > e . - , 


Tine allocated to addition i subtraction (no resrouping) for 5 students 
over 17 weeks of instruction 
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